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Abstract 

. The investor is interested in the expected return and he is also concerned about 

I the risk and the uncertainty assumed by the investment. One of the most popular 

^ ' concepts used to measure the risk and the uncertainty is the variance and/or the 

. standard-deviation. In this paper we explore the following issues: Is the standard- 

O^i deviation a good measure of risk and uncertainty? What are the potentialities of 

the entropy in this context? Can entropy present some advantages as a measure 
of uncertainty and simultaneously verify some basic assumptions of the portfolio 
management theory, namely the effect of diversification? 
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^ • This paper examines the adequacy of entropy as a measure of uncertainty in 

portfolio management in finance and its behaviour is compared with the most 
popular risk measure used in finance: the variance. 

It is quite common to relate the variance or the standard-deviation and the 
VaR (Value-at-Risk) as the main measures of risk and uncertainty in finance. 
However, some authors [see e.g. Soofi (1997)] point out that these measures 
may fail in some specific situations as measures of uncertainty, since they 
require that the underlying probability distribution is symmetric and neglect 
the possibility of extreme events such as, for example, the existence of fat-tails. 

The main goal of this paper is to assess the capability of entropy to measure 
the uncertainty in portfolio management. We describe the theoretical back- 
ground of entropy and its mathematical properties, we present a comparative 
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analysis between the entropy and the variance/standard-deviation as measures 
of uncertainty in the stock market and we discuss the empirical results of a 
comparative analysis between the CAPM and some measures of information 
theory (namely entropy, conditional entropy and mutual information). 

The results obtained point to the conclusion that the entropy observes the 
effect of diversification and is a more general measure of uncertainty than the 
variance, since it uses more information about the probability distribution. 
The mutual information and the conditional entropy show a good perfor- 
mance when compared with the systematic risk and the specific risk estimated 
through the linear Market Model. 



1 Theoretical Background 

According to Shannon (1948) entropy satisfies the main properties of a good 
measure of uncertainty. Let pi, ...,Pn be the probabilities of occurrence of a set 
of events. The entropy for discrete distributions is given by [Shannon (1948)] 
H (X) = — J2iPi^ogpi. For continuous distributions, where px{x) is the den- 
sity function of the random variable X, the entropy is given by 



Note that H (X) and H (Y) are the entropies of the random variables X & X 
and y e y , // {X, Y) is the joint entropy, and H {Y\X) and H {X\Y) are the 
conditional entropies. 

The properties of entropy for discrete and continuous distributions are mainly 
alike. In particular we have [Shannon (1948); Kraskov et al. (2004)]: (i) if 
X is limited to a certain volume v in its space, then H{X) is a maximum 
and is equal to \ogv when px{x) is constant, 1/v, in the volume; {ii) for 
any two variables X and F , we have H (X, Y) < H (X) + H {Y) , where 
the equality holds if (and only if) X and Y are statistically independent, 
i-e. Pxy^x^y) = Px{x)pY{y)] {in) the joint entropy is given by H {X^Y) ~ 



H {X)+H {Y\X) ^ H {Y) + H {X\Y) ,since H{X) + H (Y) > H {X, Y) , then 
H {Y)>H {Y\X) and H {X) > H {X\Y) . 



The entropy of a normal distribution can be parametrically estimated by 



NH {X) = log [V2^a) , where a is the standard-deviation, and tt and e are 

defined as usually. Notice that the assumption that the data follow a normal 
distribution is very common in portfolio management and regression analysis. 

The mutual information is a measure of association between variables and can 
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be defined as follows [Shannon (1948)]: 

I (X, Y)^H{Y)-H {Y\X) ^jj px,y {x, y) log ^^^^^^dxdy. (2) 

The mutual information is a nonnegative measure [Kullback (1968)], being 
equal to zero if and only if X and Y are statistically independent. In this 
way, the mutual information between two random variables X and Y can be 
seen as a measure of dependence between these variables, or even better, it 
can be regarded as a measure of the statistical correlation between X and Y . 
However, we can not say that X is causing Y or vice-versa. 

We also need to define a measure that can be directly comparable with the 
hnear correlation coefficient. In equation (2), we have < I {X,Y) < +00, 
which hampers eventual comparisons between different samples. Some authors, 
namely Granger and Lin (1994), Darbellay (1998) and Soofi (1997) have used a 
standardized measure for the mutual information, referred to as the global cor- 
relation coefficient, defined by A (X, Y) — Vl — e~'^^^-^'^\ This measure varies 
between and 1, being thus directly comparable with the linear correlation 
coefficient. 

The function A (X, Y) captures the overall linear and nonlinear dependence 
between X and Y . This measure can be used as a measure of predictability 
based on an empirical probability distribution, although it is not derived from 
any particular model of predictability. In this particular case, the above men- 
tioned properties assume the following form: (i) A (X, Y) = 0, if and only if X 
contains no information on Y; {ii) A (X, Y) = 1, if there is a perfect relation- 
ship between the vectors X and Y. This is the farthest case of determinism; 
(iii) when modelling the input-output pair (X, Y) by any model with input 
X and output U = f (X), where / is a function of X, the predictability of Y 
by U cannot exceed the predictability of Y by X, i.e., A (X, F) > A {U, Y) . 

One of the difficulties to estimate the mutual information on the basis of em- 
pirical data lies on the fact that the underlying p.d.f. is unknown. To overcome 
this problem, there are essentially three different methods to estimate the mu- 
tual information: histogram-based estimators, kernel-based estimators and 
parametric methods. ^ We will use the marginal equiquantization histogram- 
based estimation process proposed by Darbellay (1998), in order to minimize 
the bias that may occur. 

Finally, we should note that the introduction of entropy as a measure of un- 



The histogram-based estimators can be divided in two groups: equidistant cells 
and cquiprobablc cells, i.e. marginal equiquantisation. The second approach presents 
some advantages, since it allows for a better adequacy to the data and maximizes 
mutual information [Darbellay (1998)]. 
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certainty in finance goes back to Philippatos and Wilson (1972), who present 
a comparative analysis between the behaviour of the standard-deviation and 
the entropy in portfolio management. They conclude that entropy is more 
general and has some advantages relatively to the standard-deviation, such as 
Dionisio et al. (2006). 



2 Entropy and variance: a comparative analysis 

Usually, the variance is the central measure in the risk and uncertainty anal- 
ysis in financial markets. However, the entropy can be used as an alternative 
measure of dispersion, and some authors consider that the variance should 
be interpreted as a measure of uncertainty with some precaution [see, e.g. 
Maasoumi (1993) and Soofi (1997)]. 

Ebrahimi, Maasoumi and Soofi (1999) examined the role of the variance and 
entropy in ordering distributions and random prospects, and concluded that 
there is no general relationship between these measures in terms of ordering 
distributions. They found that, under certain conditions, the ordering of the 
variance and entropy is similar for transformations of continuous variables, 
and show that the entropy depends on many more parameters of a distribu- 
tion than the variance. Indeed, a Legendre series expansion shows that the 
entropy is related to higher-order moments of a distribution and thus, unlike 
the variance, could offer a better characterization of px (x) since it uses more 
information about the probability distribution than the variance [see Ebrahimi 
et al. (1999)]. 

Maasoumi and Racine (2002) argue that when the empirical probabihty distri- 
bution is not perfectly known, the entropy constitutes an alternative measure 
for assessing the uncertainty, predictability and also goodness-of-fit. Likewise, 
McCauley (2003) defends that the entropy represents the disorder and uncer- 
tainty of a stock market or a particular stock, since the entropy has the ability 
to capture the complexity of systems without requiring rigid assumptions that 
may bias the results obtained. 

Our empirical analysis is based on daily closing prices of 23 stocks rated in the 
Portuguese stock market {Euronext Lisbon)., covering the period from June, 
28, 1995 to December, 30, 2002, which corresponds to 1858 observations per 
stock in order to compute the rates of return. The stock index PSI 20 is used 
as the market benchmark, or proxy, since it is the index that better represents 
the Euronext Lisbon. A preliminary statistical analysis of the rates of return 
reveals that the null that the empirical distributions are Gaussian should be 
rejected since they show high levels of kurtosis and skewness. 
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Firstly, we perform a comparative analysis between the entropy and the loga- 
rithm of the standard-deviation, In (c), for each stock in our data set and for 
the stock index PSI 20. The entropy was computed using expression (1) mea- 
sured in nats. The In (cr) was used instead of a in order to provide a correct 
comparison between these measures. The results are shown in Figure 1. 



Fig. 1. Entropy versus In (a) . 

As may be seen, there is a strong positive relationship between the entropy 
and the In (a). There are, however, some larger deviations and the null under 
the Jarque-Bera test is rejected for all stocks and also for the stock index PSI 
20. In Figure 1 the larger deviations from the regression line correspond to 
stocks that also exhibit higher levels of skewness and kurtosis. 

We also compared the results between the empirical entropy and the normal 
entropy for each stock. The results reveal that the normal entropy takes al- 
ways higher values than the empirical entropy, indicating that the uncertainty 
of these stocks and index is smaller than that we would observe if they were 
normally distributed. Thus, we can admit evidence of some predictability of 
the rate of returns, or at least that it is higher than the one assumed by the 
financial theory. Again, the main differences between the normal entropy and 
the empirical entropy are found in the stocks that exhibit the highest levels 
of kurtosis, skewness, autocorrelation and heteroskedasticity. Therefore, the 
empirical entropy appears to be sensitive to higher-order moments of the dis- 
tribution, supplying thus more information about the stock and its probability 
distribution. ^ 



^ We also performed an analysis of the diversification effect for randomly selected 
portfolios [see Dionisio et al. (2006)]. This study revealed that both the entropy 
and the standard-deviation tend to decrease when more assets are included in the 
portfolio, pointing to the conclusion that entropy is sensitive to the effect of diver- 
sification. 
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3 Analysis of dependency between the stocks and the PSI 20 index 

In the context of portfolio management researchers tend to pay more attention 
to the systematic risk than to the specific risk, because the latter can be 
minimized (and in the limit can be zero) by an effective diversification process. 
Usually, the systematic risk in financial theory is measured by the Beta of the 
CAPM. In this context, it is assumed that the rate of returns of an asset i is 
equal to the sum of the risk free rate of return (i?/) and the compensation for 
the risk {[E (Rm) — (Rf)] Pi}- In this second term, the coefficient /3 plays an 
important role, since it measures the sensibility of the rate of returns of the 
asset (or portfolio) to the risk premium, i.e. the systematic risk. Recall that the 
variance of an asset or portfolio (cr^) can be decomposed in two components: 
the sum of squares of the regression and the residual sum of squares, that is 

= + 4, (3) 

where cr^ is the variance of the independent variable, in this case the variance 
of the market benchmark, and cr^j is the residual term of the variance that can 
be minimized through a diversification process. In order to estimate the Beta 
of the CAPM we use the Market Model that can be given by 

Rit = Rf, + [Rmt - Rft] Pi + Sit. (4) 

The Market Model is usually estimated by OLS. However, statistical tests and 
the empirical evidence show that the residuals are not white noise and thus 
OLS may not be appropriate. 

The main goal of this section is to evaluate the level of dependence between 
each stock and the stock index. The measures of information theory described 
before, namely the entropy, the conditional entropy and the mutual informa- 
tion are used to evaluate the (in) dependence between the stocks and the stock 
index PSI 20. If the residuals are white noise, then the global correlation coef- 
ficient (A) will be similar to the linear correlation coefficient (i?) and the Beta 
is a good measure of the systematic risk. However, in case of nonlinearities 
and irregularities in the behaviour of the residuals, the simple linear regression 
model is not able to capture the existence of a global relationship between the 
stocks and the stock index PSI 20. In this case, the mutual information and 
the global correlation coefficient can be potential sources of information for 
the investor. 

One may distinguish the "global" uncertainty from the "residual" uncertainty 
based on the properties of the entropy. The entropy of a stock (or any other 
variable) can be decomposed as follows 

H{X)^I(X,PSI) + H{X\PSI), (5) 
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where X represents the stock and PS I denotes the stock index. The expression 
(5) can be roughly compared with the expression (3), where the first term 
refers to the level of association or dependence between the asset (stock or 
portfolio) and the proxy PSI, and the second term is the variation of that 
asset (or stock, or portfolio) that is independent from the proxy used. 

The Beta was estimated for each stock as well as the linear correlation co- 
efficient resulting from the Market Model. In addition, we also calculate the 
systematic risk, /^^cr^, and the specific risk, cx^j. In order to provide some com- 
parisons between the two approaches, we calculated the mutual information 
between each stock and the stock index PSI 20, [/ (X, PS I)] . the conditional 
entropy [H {X\PSI)] , and the global correlation coefficient, A. 

As we can see in Figure 2, there is a positive relationship between the system- 
atic risk and the mutual information, and between the specific risk and the 
conditional entropy, although these measures are not directly comparable. 



Fig. 2. Comparative analysis between the systematic risk, Pfa"^ (continuous line), 
and the mutual information / and between the specific risk, a'^- (continuous line), 
and the conditional entropy, H {X\PSI) . 

In spite of the evidence of a positive and strong relationship between the 
variance and the measures of the theory of the information, we performed a 

comparison between measures that can be directly compared. To this end, we 
used the global correlation coefficient or lambda (A) and the global correlation 
coefficient assuming that the distribution is normal or lambda normal (A„).^ 
We found that there are stocks whose relationship with the stock index PSI 

^ The lambda normal was computed using equation the concept of normal mu- 
tual information given by: IMN {X, Y) = log (l — i?^ {X, y)) , where is the 
coefficient of determination. 
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20 exhibit strong discrepancies when analyzed from a global or purely linear 
perspective. In order to find the possible causes of such differences, several tests 
were accomplished for the residuals produced by the estimation of the linear 
Market Model, namely the Ljung-Box test, the Jarque-Bera test, the Engle 
test and the stabihty tests CUSUM and CUSUM-Q. The results of these tests 
indicate that there are precisely the residuals resulting from the application 
of the linear Market Model to stocks that appear to exhibit higher levels 
of nonlinearity that present more evidence of autocorrelation, non-normality, 
heteroskedasticity and nonstabihty, and this seems to be an indicator that 
linear analyses are possibly not enough to evaluate risk and uncertainty. 



4 Conclusions 

The results presented in this paper point to the conclusion that the entropy 
observes the effect of diversification and is a more general measure of uncer- 
tainty than the variance, since it uses more information about the probability 
distribution. The mutual information and the conditional entropy show a good 
performance when compared with the systematic risk and the specific risk es- 
timated through the linear Market Model. Nevertheless, the use of the concept 
of entropy in risk analysis and portfolio selection needs some care, because it 
does not take into account the actual values of the variables and this may 
eventually compromise its inclusion in the context of an utility function. 
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